Incorrect grad-required flags

In ForwardGroupedMLP_CuTeGEMMSwiGLU_MXFP8.fuser_forward, swiglu_ctx.input_requires_grad and swiglu_ctx.extra_input_requires_grad are set to True unconditionally (and input_requires_grad is set to requires_grad unconditionally). This will make ScaledSwiGLU.fuser_backward compute grad_input and grad_extra_input even when neither input_ nor scales require grads, which violates autograd semantics and can raise (e.g., scales.detach() passed into the fused kernel, but extra_input_requires_grad=True forces a gradient).

This should be set based on the actual requirements:

• input_requires_grad = input_.requires_grad
• swiglu_ctx.extra_input_requires_grad = scales.requires_grad
• and for FC weights, check each parameter’s requires_grad (not just weight0).

Broken fusion window scan

Both fuse_backward_ops and fuse_forward_ops have a window/shift loop that can drop or reorder ops when the pattern doesn’t match. In the non-matching branch you do out.extend(window[:-2]); window = window[-2:] and then immediately do out.extend(window[:-3]) (which is a no-op for a 2-element window) before refilling. This causes the scan to advance by 1 op in some cases and by 2 in others, and it never emits window[-1] until the very end. For sequences like [A,B,C,D] where [A,B,C] doesn’t match but [B,C,D] would (or vice versa), this loop will not correctly consider all 3-op windows and can produce an incorrect fused op list.

This needs a standard sliding-window approach (advance by 1 when not matching; replace 3->1 when matching) to ensure no ops are skipped or duplicated.

I think your reading of the code is incorrect, although I would take suggestions for improving clarity. The loop condition guarantees the window size is 3 at the beginning of each iteration. We either fuse the window or eject the first op in the window, and then we refill back up to window size 3.

The only hint of an edge case I can see is if we perform a fusion, and that fused op can participate in further fusions. Then we might want to rewind the sliding window so that we reexamine the fused op in each window position. However, we know that the fused op is final, and we can safely advance the window past it.